Video coding refers generally to coding motion picture information to transmission over a bandwidth limited channel. Various video coding techniques are known. The most common techniques, such as those are standardized in the ITU H-series and MPEG-series coding specifications, employ motion compensation prediction to reduce channel bandwidth. Motion compensated video coders exploit temporal redundancy between frames of a video sequence by predicting video content of a new frame currently being decoded with reference to video content of other frames that were previously decoded. At a decoder, having received and decoded a first number of frames, the video decoder is able to use decoded video content of the previously decoded frames to generate content of other frames.
An encoder often estimates performance of a hypothetical reference decoder (HRD) to confirm coding decisions that it makes during the coding process. The operation of an HRD is described in the H.264 standard. A simplified description of the operation of the HRD of H.264 is presented with reference to FIGS. 1 and 2. FIG. 1 illustrates a general view of the encoding and decoding process, while FIG. 2 illustrates a general block diagram of an HRD. During coding of a source video sequence, an encoder 110 refers to a HRD, a model of decoder performance. By modeling decoding performance through the HRD, the encoder 110 may estimate how its selection of coding parameters might affect decoding performance. Once the encoder has encoded the data that should satisfy performance requirements of the HRD per the H.264 standard, encoded data can be delivered via network 130 to a decoder 120. For example, the decoder can be one that is used in an iPod® or other similar device. The decoder 120 is expected to perform at least as well as the decoder represented by the HRD in terms of speed and accuracy of output. In the decoder 120, encoded data enters a coded picture buffer (CPB), is decoded by a decode process and buffered in a decoded picture buffer (DPB).
According to the standard H.264, the encoded data is assumed to be decoded instantaneously in the HRD by an instantaneous decoding process at the CPB removal time. Compliance with the standard H.264 is typically maintained in practical decoders by providing a CPB and a corresponding DPB that is larger than that required to comply with the standard. Presently, CPB size is determined by the encoder, which encodes the data, inserts size and other parameters, such as input rate, in the encoded data bitstream.
With reference to FIG. 2, a conventional HRD 200 models a video decoder that includes a CPB 210, decoder 220, and a DPB 240. Per standard H.264, the HRD models input rate of the channel into the CPB 210 as finite, but the output rate to a decoder 220 is infinite. The CPBs in practical real-world decoders will have a finite exit rate. The decoder 220 also is presumed to have an infinite output rate.
The CPB 210 as modeled by the HRD receives encoded from the channel at an input rate shown, for discussion purposes, as RIN in the figure. The input rate RIN can be included in video usability information (VUI), as discussed in the standard. More specifically, the VUI may contain “timing information” and/or “HRD parameters”. More specifically still, the timing information may specify the time between output pictures from the DPB. The HRD parameters may specify, for example, the CPB size and the maximum input bit rate. There may be several CPB size/input bitrate specifications (called “schedules”), in which case the bitstream is expected to allow correct HRD operation for each schedule. Generally, the standard H.264 basically specifies an encoder that will not overflow or underflow a CPB, if provisioned as identified in bitstream. The decoder 220 receives the encoded data from the CPB at an exit rate that is not modeled (infinity), decodes the data instantaneously via a decode process, and forwards the data at an exit rate of infinity to the decoded picture buffer (DPB). The DPB is modeled to have a finite output rate, which is shown, for discussion purposes, as ROUT in the figure. The output rate ROUT can be included in the supplemental enhancement information (SEI) in the standard. More specifically, the SEI may contain “Buffering period” messages and “Picture timing” messages for each schedule. More specifically still, the Buffering period messages may specify initial CPB removal delays both at the beginning of the video and at random access points within the video. The Picture timing messages may specify CPB removal delay and DPB removal delay on a per picture basis.
Real world decoders 220 can be either hardware or software devices. In hardware decoders, the CPB 210 is commonly a memory component whose size is predetermined at the time of manufacture. In software (SW) decoders, the CPB 210 is commonly a span of memory allocated from a larger memory space. Software latency can cause input rates and exit rates to vary instantaneously even though overall input and exit rates may be steady.
Since present encoders use the H.264 standard HRD to model a CPB having a finite input rate and a finite size but an infinite exit rate, the HRD can artificially lower the size required for CPBs to buffer the coded video data. Often, decoder designers will make accommodation for CPBs that are much larger, i.e., have a memory capacity greater than necessary to ensure overflow/underflow events, as specified by the H.264 standard, do not occur. Of course, a designer can specify their own buffer accommodations, overflow/underflow events, or other coding/decoding parameters as desired.
The CPB of HRD 200, although having a finite input bitrate, has a theoretical infinite output bitrate, which is impractical. There is a need to implement a system that allows for modeling the output bitrate of the CPB to a practical limit. A practical limit on the output bitrate allows for more efficient allocation of buffer and decoder resources for decoding picture data for output.